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Abstract 

The pressing need for efficient compression schemes for XML documents has recently 
been focused on stack computation [6l [9], and in particular calls for a formulation of 
information-lossless stack or pushdown compressors that allows a formal analysis of their 
performance and a more ambitious use of the stack in XML compression, where so far 
it is mainly connected to parsing mechanisms. In this paper we introduce the model of 
pushdown compressor, based on pushdown transducers that compute a single injective 
function while keeping the widest generality regarding stack computation. 

The celebrated Lempel-Ziv algorithm LZ78 (TU] was introduced as a general purpose 
compression algorithm that outperforms finite-state compressors on all sequences. We 
compare the performance of the Lempel-Ziv algorithm with that of the pushdown com- 
pressors, or compression algorithms that can be implemented with a pushdown transducer. 
This comparison is made without any a priori assumption on the data's source and consid- 
ering the asymptotic compression ratio for infinite sequences. We prove that Lempel-Ziv 
is incomparable with pushdown compressors. 

Keywords 

Finite-state compression, Lempel-Ziv algorithm, pumping-lemma, pushdown compression, 
XML document. 



1 Introduction 

The celebrated result of Lempel and Ziv [lOj that their algorithm is asymptotically better 
than any finite-state compressor is one of the major theoretical justifications of this widely 
used algorithm. However, until recently the natural extension of finite-state to pushdown 
compressors has received much less attention. 

XML is rapidly becoming a standard for the creation and parsing of documents, however, 
a significant disadvantage is document size, even more since present day XML databases are 
massive. Since 1999 the design of new compression schemes for XML is an active area where 
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the use of syntax directed compression is specially adequate, that is, compression performed 
with stack memory [6, 9J. 

On the other hand the work done on stack transducers has been basic and very connected 
to parsing mechanisms. Transducers were initially considered by Ginsburg and Rose in [3] 
for language generation, further corrected in [5], and summarized in [T]. For these models 
the role of nondeterminism is specially useful in the concept of A-rule, that is a transition in 
which a symbol is popped from the stack without reading any input symbol. 

In this paper we introduce the concept of pushdown compressor as the most general stack 
transducer that is compatible with information-lossless compression. We allow the full power 
of A-rules while having a deterministic (unambiguous) model. The existence of endmarkers 
is discussed, since it allows the compressor to move away from mere prefix extension by 
exploiting A-rules. 

The widely-used Lempel-Ziv algorithm LZ78 [10] was introduced as a general purpose com- 
pression algorithm that outperforms finite-sate compressors on all sequences when considering 
the asymptotic compression ratio. This means that for infinite sequences, the algorithm at- 
tains the (a posteriori) finite state or block entropy. If we consider an ergodic source, the 
Lempel-Ziv compression coincides exactly with the entropy of the source with high probability 
on finite inputs. This second result is useful when the data source is known, whereas it is not 
very informative for general inputs, specially for the case of infinite sequences (notice that an 
infinite sequence is Lempel-Ziv incompressible with probability one). For the comparison of 
compression algorithms on general sequences, either an experimental or a formal approach is 
needed, such as that used in [8]. In this paper we follow [8] using a worse case approach, that 
is, we consider asymptotic performance on every infinite sequence. 

We compare the performance of the Lempel-Ziv algorithm with that of the pushdown- 
compressors, or compression algorithms that can be implemented with a pushdown trans- 
ducer. This comparison is made without any a priori assumption on the data's source and 
considering the asymptotic compression ratio for infinite sequences. 

We prove that Lempel-Ziv compresses optimally a sequence that no pushdown transducer 
compresses at all, that is, the Lempel-Ziv and pushdown compression ratios of this sequence 
are and 1, respectively. For this result, we develop a powerful nontrivial pumping- lemma, 
that has independent interest since it deals with families of pushdown transducers, while 
known pumping- lemmas are restricted to recognizing devices [1]. 

In fact, Lempel-Ziv and pushdown compressing algorithms are incomparable, since we con- 
struct a sequence that is very close to being Lempel-Ziv incompressible while the pushdown 
compression ratio is at most one half. While Lempel-Ziv is universal for finite-state com- 
pressors, our theorem implies a strong non-universality result for Lempel-Ziv and pushdown 
compressors. 

The paper is organized as follows. Section 2 contains some preliminaries. In section [3l we 
present our model of pushdown compressor with its basic properties and notation. In section 
m we show that there is a sequence on which Lempel-Ziv outperforms pushdown compressors 
and in section [S] we show that Lempel-Ziv and pushdown compression are incomparable. We 
finish with a brief discussion of connections and consequences of these results for dimension 
and prediction algorithms. 

Our proofs appear in the appendix. 
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2 Preliminaries 



We write Z for the set of all integers, N for the set of all nonnegative integers and Z+ for the 
set of all positive integers. Let S be a finite alphabet, with |S| > 2. T,* denotes the set of 
finite strings, and the set of infinite sequences. We write \w\ for the length of a string 
w in S*. The empty string is denoted by A. For S £ and i,j £ N, we write 5'[i..j] for 
the string consisting of the z*^ through j^^ bits of S, with the convention that 5'[i..j] = A if 
i > j, and ^[O] is the leftmost bit of S. We write S[i] for S[i..i] (the i^^ bit of S). For w £ H* 
and S G T,°°, we write w Q S if w is a prefix of S, i.e., if w = S[0..\w\ — 1]. Unless otherwise 
specified, logarithms are taken in base For a string x, denotes x written in reverse 
order. We use f{x) =_L to denote that function / is undefined on x. 

Let us give a brief description of the Lempel-Ziv (LZ) algorithm [TOj. Given an input 
X G S*, LZ parses x in different phrases Xi, i.e., x = xiX2 . . . x„ (xj G S*) such that every 
prefix y C Xi, appears before Xi in the parsing (i.e. there exists j < i s.t. xj = y). Therefore 
for every i, Xi = XK^^^bi for < i and bi £ T,. We sometimes denote the number of phrases 
in the parsing of x as P{x). 

LZ encodes Xi by a prefix free encoding of l{i) and the symbol hi, that is, if x = xiX2 ■ ■ - Xn 
as before, the output of LZ on input x is 

LZ{x) = C/(i)6iQ(2)62 . . . Q(„)6„ 

where Cj is a prefix- free coding of i (and xq = X). 

LZ is usually restricted to the binary alphabet, but the description above is valid for any 



S. 



For a sequence S G T,°°, the LZ infinitely often compression ratio is given by 

y ■ , \LZ{S[0...n-l])\ 

pLz{S) = hmmf -— . 

n^oo nlogadSI) 

We also consider the almost everywhere compression ratio 

|LZ(S[0...n- 1])| 



Rlz{S) =limsup- 



nlog2(|S|) 

3 Pushdown compression 

Definition. A pushdown compressor {PDC) is a 7-tuple 

C = {Q,J:,r,6,u,qo,zo) 

where 

• E is the finite input alphabet 

• Q is a finite set of states 

• r is the finite stack alphabet 

• (5 : Q X (S U {A}) X r — > Q X r* is the transition function 

• : Q X (S U {A}) X r ^ S* is the output function 
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• ^Zo G Q is the initial state 

• Zo G r is the start stack symbol 

We write 6 = {5q , Sr* ) • Note that the transition function 5 accepts A as an input character 
in addition to elements of S, which means that C has the option of not reading an input 
character while altering the stack. In this case S{q,X,a) = (g',A), that is, we pop the top 
symbol of the stack. To enforce determinism, we require that at least one of the following 
hold for sdl q £ Q and a G T: 

• 5{q, A, a) =_L 

• 6{q, b, a) =_L for all 6 G S 

We restrict 6 so that zq cannot be removed from the stack bottom, that is, for every q £ Q, 
6 E S U {A}, either d{q, b, zq) =_L, or 6{q, b, zq) = {q', vzq), where q' £ Q and v G T*. 

There are several natural variants for the model of pushdown transducer [ij , both allowing 
different degrees of nondeterminism and computing partial (multi) functions by requiring final 
state or empty stack termination conditions. Our purpose is to compute a total and well- 
defined (single valued) function in order to consider general-purpose, information-lossless 
compressors. 

Notice that we have not required here or in what follows that the computation should 
be invertible by another pushdown transducer, which is a natural requirement for practical 
compression schemes. Nevertheless the unambiguity condition of a single computation per 
input gives as a natural upper bound on invertibility. 

We use the extended transition function 6* : Q x T,* x r+ Q x T* , defined recursively 
as follows. For q G Q, v G , w £ S*, and b £ T, 

S*(qXv)=l '^**^'^'3(9''^''^)''^''^r*(g,A,^;)), if 6{q, X,v) 
I (g, f), otherwise. 



5*{q,'wb,v) 



6*{6{6*{q,w,v),b),X), if 6*{q,w,v) /± and 6{6* {q,w,v),b) 
_L, otherwise. 



That is, A-rules are inside the definition of 6*. We abbreviate 6* to 5, and 5{qo,w,zo) to 
5{w). We define the output from state q on input w £ Ti* with z £T* on the top of the stack 
by the recursion iy{q, A, z) = A, 

u{q, wb, z) = u{q, w, z)u{6Q{q, w, z), b, 6r* (q, w, z)). 

The output of the compressor C on input w £ T,* is the string C{w) = i'{qQ,w, zq). 

The input of an information-lossless compressor can be reconstructed from the output and 
the final state reached on that input. 

Definition. A PDC C = {Q,T,,T,5,i',qo, zq) is information-lossless (IL) if the function 

S* ^ S* X Q 

w {C{w),Sq{w)) 
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is one-to-one. An information-lossless pushdown compressor (ILPDC) is a PDC that is IL. 
Intuitively, a PDC compresses a string w if |C(?i;)| is significantly less than \w\. Of course, if 
C is IL, then not all strings can be compressed. Our interest here is in the degree (if any) to 
which the prefixes of a given sequence S G can be compressed by an ILPDC. 
Definition. If C is a PDC and S G , then the compression ratio of C on S is 

n^(Q\- liTv, inf \C{S[0..n-l])\ 
pc{b)- hniinf „iog^(|s|) 

Definition. The pushdown compression ratio of a sequence S G S°° is 

Ppd{S) = M{pc{S) I C is an ILPDC} 

We can consider dual concepts Rc and Rpo i>y replacing liminf with limsup in the previous 
definition. 



3.1 Endmarkers and pushdown compression 

Two possibilities occur when dealing with transducers on finite words: should the end of the 
input be marked with a particular symbol # or not? As we will see, this is a rather subtle 
question. First remark that both approaches are natural: on the one hand, usual finite state 
or pushdown acceptors do not know (and do not need to know) when they reach the end 
of the word; on the other hand, everyday compression algorithms usually know (or at least 
are able to know) where the end of the input file takes place. For a word w, we will denote 
by C{w) the output of a transducer C without endmarker, and C{w#) the output with an 
endmarker. 

Unlike acceptors, transducers can take advantage of an endmarker: they can indeed output 
more symbols when they reach the end of the input word if it is marked with a particular 
symbol. This is therefore a more general model of transducers which, in particular, does not 
have the strong restriction of prefix extension: if there is no endmarker and C is a transducer, 
then for all words wi,W2, wi ^ W2 ^ C{wi) C C{w2)- Let us see how this restriction limits 
the compression ratio. 

Lemma 3.1 Let C be an IL pushdown compressor with k states and working with no end- 
marker. Then on every word w of size \w\ > k, the compression ratio of C is 

\C{w)\ ^ 1 
\w\ - 2k' 

Proof. Due to the injectivity condition, we can show that C has to output at least one symbol 
every k input symbols. Suppose on the contrary that there are words t, u, with \u\ = k, such 
that C does not output any symbol when reading u on input w = tu. Then all the k + \ 
words t and te[0..i] for < i < — 1 have the same output by C, and furthermore two of 
them have the same final state because there are only k states. This contradicts injectivity. 
Thus C must output at least one symbol every k symbols, which proves the lemma. 

□ 

This limitation does not occur with endmarkers, as the following lemma shows. 
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Lemma 3.2 For every k, there exists an IL pushdown compressor C with k states, working 
with endmarkers, such that the compression ratio of C on 0" tends to 1/k'^ when n tends to 
infinity, that is, 

n^oo n K 

Proof, [sketch] On input 0", our compressor outputs (roughly) 0"^^^ as follows: by selecting 
one symbol out of each k of the input word (counting modulo k thanks to k states), it pushes 
Qu/k i-j-^g stack. Then at the end of the word, it pops the stack and outputs one symbol 
every k. Thus the output is 0"/*^ . 

To ensure injectivity, if the input word w is not of the form 0" (that is, if it contains a 1), 
then C outputs w. 

□ 

It is worth noticing that it is the injectivity condition that makes this computation impossible 
without endmarkers, because one cannot decide a priori whether the input word contains a 
1 . Thus pushdown compressors with endmarkers do not have the limitation of Lemma 13.11 
Still, as Corollary 14. 51 will show, pushdown compressors with endmarkers are not universal for 
finite state compressors, in the sense that a single pushdown compressor cannot be as good 
as any finite state compressor. 

It is open whether pushdown compressors with endmarkers are strictly better than with- 
out, in the sense of the following question. 

Open question. Do there exist an infinite sequence S, a constant < a < 1 and an IL 
pushdown compressor C working with endmarkers, such that pc{S) < a, but pc'{S) > a, for 
every C' IL pushdown compressor working without endmarkers? 

In the rest of the paper we consider both variants of compression, with and without 
endmarkers. We use the weakest variant for positive results and the strongest for negative 
ones, therefore showing stronger separations. 



4 Lempel-Ziv outperforms Pushdown transducers 

In this section we show the existence of an infinite sequence S E {0, 1}°° whose Lempel- 
Ziv almost everywhere compression ratio is but for any IL pushdown compressor (even 
working with endmarkers) the infinitely often compression ratio is 1. The rough idea is that 
Lempel-Ziv compresses repetitions very well, whereas, if the repeated word is well chosen, 
pushdown compressors perform very poorly. We first show the claim on Lempel-Ziv and then 
prove a pumping-lemma for pushdown transducers in order to deal with the case of pushdown 
compressors. 



4.1 Lempel-Ziv on periodic inputs 

The sequence we will build consists of regions where the same pattern is repeated several 
times. This ensures that Lempel-Ziv algorithm compresses the sequence, as shown by the 
following lemmas. 

We begin with finite words: Lempel-Ziv compresses well words of the form in". The idea 
is that the dictionary remains small during the execution of the algorithm because there are 
few different subwords of same length in tn" due to the period of size The statement is 
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slightly more elaborated because we want to use it in the proof of Theorem 14.21 where we will 
need to consider the execution of Lempel-Ziv on a nonempty dictionary. 

Lemma 4.1 Let n £ N and let t, u, £ S*, where u ^ X. Define / = 1 + |t| + |u| and Wn = tu^ . 
Suppose we want to run Lempel-Ziv on Wn, but possibly, another word has already been parsed 
so that the dictionary of phrases is possibly not empty and already contains d > phrases. 
Then we have that 

\LZ{Wn)\ ^ y/2l\Wn\log{d+y/2l\Wn\) 
\Wn\ ~ \Wn\ 

This leads us to the following lemma on a particular infinite sequence. 

Theorem 4.2 (LZ compressibility of repetitive sequences) Let (tj)j>i and (nj)j>i be 
sequences of words, where / A, Vz > 1. Let (nj)j>i be a sequence of integers. Let S be the 
sequence defined by 

S = tiu'lH2ulH^ul^ . . . 
If the sequence {ni)i>i grows sufficiently fast, then 

Rlz{S) = 0. 

4.2 Pumping-lemma for injective pushdown transducers 

This section is devoted to the statement and proof of a pumping-lemma for pushdown trans- 
ducers. In the usual setting of recognition of formal languages by pushdown automata, the 
pumping-lemma comes from the equivalence between context-free grammars and pushdown 
automata, see for instance [11]. However, the proof is much less straightforward without 
grammars, as is our case since we deal with transducers and not acceptors. Moreover, there 
are three further difficulties: ffi'st, we have to consider what happens at the end of the word, 
after the endmarker (where the transducer can still output symbols when emptying the stack); 
second, we need a lowerbound on the size of the pumping part, that is, we need to pump on 
a sufficiently large part of the word; third, we need the lemma for an arbitrary finite family 
of automata, and not only one automaton. All this makes the statement and the proof much 
more involved than in the usual language-recognition framework (see the appendix for all 
details). 

Lemma 4.3 (Pumping-lemma) Let T be a finite family of ILPDC. There exist two con- 
stants a, /3 > such that \/w, there exist t,u,v £ S* such that w = tuv satisfying: 

• \u\ 

• yC gJ", ifC{tuv) = xyz, then C{tu'^) = xy", Vn G N. 
Taking into account endmarkers, we obtain the following corollary: 

Corollary 4.4 (Pumping-lemma with endmarkers) Let T be a finite family of ILPDC. 
There exist two constants a, /3 > such that every word w can be cut in three pieces w = tuv 
satisfying: 

1. \u\ > [a\wf\; 
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2. there is an integer c > such that for all C £ J^, there exist five words x, x' ,y,y' , z such 
that for all n > c, C{tu"'v^) = xy"-zy'"~'^x' . 

Let us state an immediate corollary concerning universality: pushdown compressors, even 
with endmarkers, cannot be universal for finite state compressors, in the sense that the 
compression ratio of a particular pushdown compressor cannot be always better than the 
compression ratio of every finite state compressor. 

Corollary 4.5 Let C be an IL pushdown compressor (with endmarkers). Then pc{0°°) > 0. 
In particular, no pushdown compressor is universal for finite state compressors. 

Proof. By Corollary 14.41 there exist two integers k,k', {k' > 1), a constant c > and five 
words x,x' ,y,y' , z such that for all n > c, C(0'^0'^"#) = xy"^ zy''^~'^x' . By injectivity of C, y 
and y' cannot be both empty. Hence the size of the compression of O'^O'^ " is linear in n. This 
proves the first assertion. 

Since for every e > there exists an IL finite state compressor C such that Rc'{^°°) < e, 
the pushdown compressor C cannot be universal for finite state compressors. 

□ 

4.3 A pushdown incompressible sequence 

We now show that some sequences with repetitions cannot be compressed by pushdown 
compressors. We start by analyzing the performance of PDC on the factors of a Kolmogorov- 
random word. This result is valid even with endmarkers. 

Lemma 4.6 For every T finite family of ILPDC with k states and for every constant e > 0, 
there exists Mjf,^ G N such that, for any Kolmogorov random word w = tu, if\u\ > Mjr^^ log 
then the compression ratio for C £ of u on input w is 

\C{tu)\ - \C{t)\ ^ ^ ^ 
\u\ ~ 

We can now build an infinite sequence of the form required in Theorem 14.21 that cannot 
be compressed by bounded pushdown automata. The idea of the proof is as follows: by 
Corollary 14.41 in any word w we can repeat a big part uolw while ensuring that the behaviour 
of the transducer on every copy of u is the same. If u is not compressible, the output will be 
of size almost therefore with a large number of repetitions the compression ratio is almost 
1. 

Theorem 4.7 (A pushdown incompressible repetitive sequence) LetT, be a finite al- 
phabet. There exist sequences of words {tk)k>i ^.n-d {uk)k>i, where 7^ A, V/c > 1, such that 
for every sequence of integers {nk)k>i growing sufficiently fast, the infinite string S defined 
by 

S = tin"^t2^2^i3%^ • • • 

verifies that 

PciS) = 1, 

VC G ILPDC (without endmarkers). 
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Combining it with Theorem 14.21 we obtain the main result of this section, there are sequences 
that Lempel-Ziv compresses optimahy on almost every prefix, whereas no pushdown com- 
presses them at all, even on infinitely many prefixes and using endmarkers. 

Theorem 4.8 There exists a sequence S such that 

Rlz{S) = 

and 

Pc{S) = 1 

for any C S I LP DC (without endmarkers). 

The situation with endmarkers is slightly more complicated, but using Corollary 14.41 (the 
pumping lemma with endmarkers) and a similar construction as Theorem 14.71 we obtain the 
following result. Note that we now use the limsup of the compression ratio for ILPDC with 
endmarkers. 

Theorem 4.9 There exists a sequence S such that 

Rlz{S) = 

and 

RciS) = 1 
for any C € ILPDC (using endmarkers). 

5 Lempel-Ziv is not universal for Pushdown compressors 

It is well known that LZ |10j yields a lower bound on the finite-state compression of a sequence 
|10j . ie, LZ is universal for finite-state compressors. 

The following result shows that this is not true for pushdown compression, in a strong 
sense: we construct a sequence S that is infinitely often incompressible by LZ, but that has 
almost everywhere pushdown compression ratio less than ^. 

Theorem 5.1 For every m G N, there is a sequence S G {0, 1}°° such that 

PLZ{S) > 1 - - 

m 

and 

Rpd{S) < ^. 

The proof of this result is included in the appendix. 
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6 Conclusion 



The equivalence of compression ratio, effective dimension, and log-loss unpredictability has 
been explored in different settings [21 [71 [13]. It is known that for the cases of finite-state, 
polynomial-space, recursive, and constructive resource-bounds, natural definitions of com- 
pression and dimension coincide, both in the case of infinitely often compression, related 
to effective versions of Hausdorff dimension, and that of almost everywhere compression, 
matched with packing dimension. The general matter of transformation of compressors in 
predictors and vice versa is widely studied [H] . 

In this paper we have done a complete comparison of pushdown compression and LZ- 
compression. It is straightforward to construct a prediction algorithm based on Lempel-Ziv 
compressor that uses similar computing resources, and it is clear that finite-state compres- 
sion is always at least pushdown compression. This leaves us with the natural open question 
of whether each pushdown compressor can be transformed into a pushdown prediction al- 
gorithm, for which the log-loss unpredictability coincides with the compression ratio of the 
initial compressor, that is, whether the natural concept of pushdown dimension defined in [3] 
coincides with pushdown compressibility. A positive answer would get pushdown computa- 
tion closer to finite-state devices, and a negative one would make it closer to polynomial-time 
algorithms, for which the answer is likely to be negative [12]. 

Acknowledgments. The authors thank Victor Poupet for his help on the proof of 
Lemma 14.31 
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Technical Appendix 



A Proofs of Section 13.11 

Proof of Lemma 13.21 On input 0*^, the output of our pushdown compressor C will be 
C(0"#) = 01*0'-"'/'^ Jp, where i + kj = n — k'^\n/k'^\ , whereas on any other word w (that is, 
any word that contains a 1), the output will be Iw. This ensures the injectivity condition. 

The compressor C works as follows: while it reads zeroes, it pushes on the stack one every 
k zeroes. This is done with k states by counting modulo k, the state i meaning that the 
number of zeroes up to now is i modulo k. 

If the endmarker ^ is reached in state i and only zeroes have been read, then C first 
outputs 01*. Then it begins popping the stack symbol after symbol: it outputs one every 
k zeroes by counting again modulo k. At the end of the stack, if it is in state j, it finally 
outputs P. Thus the output is C(0''#) = 01'0L"/'='Jp where i + kj = n - k'^[n/k'^\. 

Otherwise, when C reads a 1 in state it outputs one 1 and completely pops the stack 
while outputting k zeroes at each pop, then outputs 0*~^1, and then outputs the remaining 
part of the input word. Thus the output is C(w#) = Iw as claimed at the beginning of this 
proof. 

Thus C is injective, has k states, and verifies that lim ^^^^ = -r^. 

□ 



B Proofs of Section 14.11 

Proof of Lemma 14.11 Let us fix n and consider the execution of Lempel-Ziv algorithm on 
Wn- as it parses the word, it enlarges its dictionary of phrases. Fix an integer k and let us 
bound the number of new words of size k in the dictionary. As the algorithm parses t only, it 
can find at most \t\/k different words of size k. Afterwards, there is at most one more word 
lying both on t and the rest of Wn- 

Then, due to its period of size the number of different words of size k in u** is at 
most \u\ (at most one beginning at each symbol of u). Therefore we obtain a total of at most 
^ + 1 + |n| different new words of size k in Wn- This total is upper bounded by / = 1 + |t| + \u\ . 

Therefore at the end of the algorithm and for all k, the dictionary contains at most I new 
words of size k. We can now upper bound the size of the compressed image of Wn- Let p be 
the number of new phrases in the parsing made by Lempel-Ziv algorithm. The size of the 
compression is then plog{p + d): indeed, the encoding of each phrase consists in a new bit 
and a pointer towards one of the p + d words of the dictionary. The only remaining step is 
thus to evaluate the number p of new words in the dictionary. 

Let us order the words of the dictionary by increasing length and call ti the total length 
of the first / words (that is, the I smallest words), t2 the total length of the / following words 
(that is, words of index between / + 1 and 21 in the order), and so on: tk is the cumulated 
size of the words of index between (k — 1)/ + 1 and kl. Since the sum of the size of all these 
words is equal to \wn\, we have 

\wn\ = y^jk, 

k>l 
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and furthermore, since for each k there are at most / = 1 + |t| + |n| new words of size k, we 
have tk > kl. Thus we obtain 

p/l 2 
k>l k=l 



Hence p satisfies 



p2 



2^ < l^nl, that is, p < '\/2l\w„ 



The size of the compression of Wn is plog{p + d) < y^2l\wn\ log(d + y^2l\wn\), so 



\LZiwn)\ ^ V2/Mlog(d+ 



□ 



Remark B.l We have that 

\LZ{wn)\ _ ^flogn 



Proof of Theorem 14.21 Let us call Wi = tiU^^ and Zi = wiW2 ■ ■ - Wi. Without loss of 
generality, one can assume (for a technical reason that will become clear later) that for all i, 
\zi-i\ is big enough so that for all j > 0, 



~ — -logdzi^il + y2/i|tiu^|) 2i-2 

'^''"'^ \^~^\ - WTTy 

where = 1 + + By induction on i, let us show that we can furthermore choose the 
integers rii so that for all i: 

\LZ{z,)\ ^ J2^, 

\Zi\ - 

. for all 7 < n,-, "'^^"-^^'"'^l < 4t 



This is clear for i = 1. For z > 1, the dictionary after Zi-i has at most l^^i-il words. By 
Lemma l4.ll and by induction, the compression ratio of Zi is 



\LZ{zi)\ ^ 2\zi^i\/i + ^^2li\wi\ log(|zj-i| + ^/2li\wi\) 



where /j = 1 + \ti\ + \ui\. By taking rii sufficiently large, this can be made less or equal than 
and the first point of the induction follows. 

For the second point, for the same reasons the compression ratio of Zj_itju] is 



|LZ(zi_itin|)| ^ 2\zi^i\/i + ^2k\tiU^^\og{\zi.i\ + ^2li\tiul\ 



hi-itiu'A \zi-itiu'A 



2 I — \ + \l 2li\tiul 

-T + \J 2li\tiu'-\ 



Zi-ltiU\\ 
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Since we have supposed without loss of generahty that 



,~ ~}og{\zi-i\ + J 2li\tiul\) 2i-2 

we conclude that the compression ratio is less or equal than as required. Therefore the 
two points of the induction are proved. 
This enables us to conclude that 

RLziS) = 0. 

□ 



C Proofs of Lemma 14.3 

This section is devoted to the proof of Lemma 14.31 We first need several natural definitions. 
Definition. Let C be a PDC and w eT.*. 

• The configuration of C on t;; in the i-th symbol is the pair (g, Zi ■ ■ ■ Z/^) where q £ Q and 
Zi, . . . , Zfc G r are the state and the stack content when C has read i input symbols of 
w (hence the last transition is not a A-rule). Each configuration will be called column. 

• The partial configuration of a column (q, Z\ - ■ ■ Z/.) is {q, Zi), that is, the state and the 
top stack symbol of the configuration. 

• The diagram of the evolution of C on ti; is the sequence of all the successive columns of 
C on the input w. 

Definition. Let C be a PDC, w £ 'E,* and c a column in the diagram of C on input w. 

• The birth of c is the index of the column in the diagram. 

• The death of c is the index of the first column c' appeared after c in the diagram such 
that its stack size is strictly smaller than the stack size of c. 

• The lifetime of c {life{c)) is the difference between its death and its birth. 
Definition. Let C be a PDC, w £Ti* and c a column in the diagram of C on input w. 

• The children of c are defined as follows: let Yi, . . . ,Yj the symbols pushed on the stack 
when a symbol (having c as configuration) is read . If j = 0, then c has no child. 
Otherwise, if it exists, the first child ci is the first column in the lifetime of c; the 
second child C2 is the column where ci dies (that is, the first column where the stack 
is strictly smaller than in ci) if it happens during the lifetime of c; the third child C3 is 
the column where C2 dies if it happens during the lifetime of c, and so on. In particular, 
c has at most j children because the size of the stack in Cj+i is strictly smaller than in 
Cj, and the death of a child always happens before the death of c. 

• A descendant of a column is either a child or a child of a descendant. 
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The following definition will be useful in order to repeat a part of the diagram (that is, to 
'pump'). 

Definition. Let C be a PDC and c,d two columns in the diagram of C on input ^i; S S*. 
Then c and d are said to be equivalent if one is a descendant of the other and they have the 
same partial configuration. In particular, if we denote by u the input word between c and d, 
then for all i > 1, after reading we will obtain another column e equivalent to c and d. 
Moreover, we need some definitions regarding a finite family of compressors. 
Definition. Let be a finite family of PDC. 

• Let ci, . . . , c\jr\^ be the columns of all C G .7-" at a given input symbol (that is, with the 
same index). Then the tuple (ci, . . . , C|jp|) is called a generalized column of T. 

• Let gi, . . . , be the partial configurations of all C G at a given input symbol. Then 
the tuple (gi, . . . , q\j^\) is called a generalized partial configuration (gpc for short) of J^. 

• Two generalized columns are equivalent if for all C & the corresponding columns are 
equivalent. In particular, the gpc are the same. 

• The lifetime of a generalized column {life{{ci, . . . , C|;f|))) is the minimum of the lifetimes 
of the corresponding columns of all C ^ J-. 

We can now begin the proof of Lemma 14.31 In order to be able to pump, we need to find 
two equivalent gpc. Since the family T of transducers is finite, there is a finite number of 
transition rules, therefore let k be the maximum size of a word pushed on the stack by one 
rule. The integer p will denote the total number of gpc, that is, the product over all T G JT 
of the number of states of T multiplied by the size of the stack alphabet of T. Finally, d will 
be an integer representing a distance. 

We will upperbound the size of a counterexample, that is, the size of a word in which no 
pair of generalized columns at distance more than or equal to d are equivalent. More precisely, 
let L{p, k, d) be the maximum lifetime of a generalized column during which: 

• only p distinct gpc appear; 

• the size of a word pushed on the stack by one rule is less than or equal to k] 

• no pair of equivalent generalized columns are at distance more than or equal to d. 
Let us upperbound L(p, A;, d) using an induction. 

Claim C.l For all k,den, 

L{l,k,d) < d. 

Proof. There is only one possible gpc. If the lifetime of a generalized columns was greater 
than or equal to d, then there would be two equivalent generalized columns at distance at 
least d. 

□ 

Claim C.2 \fp,k,dG N, 

L{p +l,k,d) <d + \T\kdL{p, k, d). 
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Proof. Let (ci, . . . ,C|;f|) be a generalized column of the diagram of J- on w whose gpc is 
(gi, . . . , q\j^\), and let us upperbound its lifetime. During this lifetime, the first d generalized 
columns can be arbitrary but the remaining ones cannot contain the gpc (gi, . . . , q\j^\) since 
they are descendant of (ci, . . . ,c\j^\) (otherwise, we would obtain two equivalent generalized 
columns at distance more than or equal to d, which is not possible). Hence every generalized 
column at distance at least d from (ci, . . . , C|j^|) has its lifetime bounded by L{p, k, d). 

Let C ^ J-. For convenience, we count the size of the stack relatively to that in (ci, . . . , c\j^\), 
i.e, we will say that the size of the stack in (ci, . . . ,C|^|) is 0. Then the stack after the first 
d columns is of size at most dk, because every rule can push at most k symbols each time. 
We call the column of C at distance d of (ci, . . . , c\jr\)^ cc^ the column of C whose birth 
coincides with the death of cci , ccg the column whose birth coincides with the death of ; 
and so on. The intervals between and cc^^ between cc^ and cq, etc, will be referred to 
as 'disjoint intervals'. Remark that the size of the stack in cc-^^ is less than the size in cc*. 
(because the top symbol in cc^ has been popped). Hence the number of these columns is at 
most dk (because the size of the stack in cc^ was at most dk). 

Back to the whole family T , this enables us to bound the number of disjoint generalized 
intervals at distance at least d from (ci, . . . , C|;f|). Indeed, since every compressor has at most 
dk disjoint intervals, then there are at most \T\kd disjoint generalized intervals at distance 
at least d from (ci, . . . , c\jr\) (because the definition of the lifetime of a generalized column is 
the minimum of the lifetimes of each column). Since each of them has its lifetime bounded 
by L(p, A;, d), this proves the claim. 

□ 

Claim C.3 \/p,k,de N, 

L{p,k,d) < (|.F|/£d)P+^ 
Proof. Iterating claim KT^ we obtain that 

Lip,k,d) < dj^rnkdy = d^^^^^ 



□ 



Proof of Lemma 14.31 For the finite family J^, we have fixed constants p and k. Now, take 
a word w and let 



d = 

Then, by using Claim [231 we obtain that 
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L{p, k, d) < \w\, 

so on w there are two equivalent generalized columns at distance more than or equal to d. 
We call u the part of the input read between these two generalized columns {\u\ > d, that is, 
\u\ > [a\w\^ \ for some constants a,/3 > 0), t the part of the input previous to u and v the 
part following u. We have that 

C{tuv) = xyz 

VC G J-. Then, C{tu"') = xy" Vre G N and VC G J^. This concludes the proof of Lemma 14.31 



5 



□ 

Proof of Corollary 14.41 The proof is very similar as that of Lemma 14.31 but we have to 
take into account the computation after the endmarker Remark that an ILPDC behaves 
like a finite state transducer after because it can only read the stack and pop its top 
symbol. We therefore only have to take into account the state of the transducer in this phase. 
Thus, instead of merely looking for two equivalent generalized partial configurations as in the 
proof of Lemma 14. 3t we are looking for two equivalent generalized partial configurations for 
which the "corresponding state" after ^ while popping the stack is the same. The argument 
then goes through by considering pairs (gpc, state), but now v plays a role as being the word 
influencing the topmost symbols of the stack. 

□ 



D Proofs of Section 14.3 



Lemma 14.61 follows directly from this lemma. 

Lemma D.l Let w G S* he a Kolmogorov random word (that is, w is such that K{w) > \w\) 
and C he an ILPDC with k states. Let \C\ he the size of its encoding (i.e. the complete 
description of C ). For every t,u £ Ti* such that w = tu 

\C{tu)\ - \C{t)\ > \u\ - \C\ -logA;-21og|u;| -21og|(:7| - 2 log log A; - 3. 

Proof of Lemma ID. 11 Given w, since C is IL, w can be recovered from t, C, i^{u) and the 
final state q oi C on input w. If we encode a tuple (xi, . . . , Xm) as 

where = is in binary, then encoding the 4-tuple (t, C, i^{u), q) takes size 

2[log|tn + l + +2[log|Cn + +2[logriogA;]] +1 + [log A;] +W{u)\, 

where k is the number of states of C. We therefore obtain 

\w\ < K{w) < 21og|t| + |t| + 21og|C| + \C\ +21oglogA; + logA; + +0(1) 

and the lemma follows. 

□ 

Proof of Theorem 14.71 For ah k, let J^k be the (finite) family of ILPDC with at 
most k states and with a stack alphabet of size k. Let and Pk be the constants given by 
Corollary 14.41 for the family J^^- Let = Mjr^^i/f. as given by Lemma 14.61 

For all k, take a Kolmogorov-random word Wk of size big enough so that [aA:|wfc|^'=J > 
Mklog\wk\. By Corollarv 14.41 can be cut in three pieces = tkU^Vk such that \uk\ > 
Mklog\wk\. For all A;, define to be the least integer so that (l — l/fc)|M^* I > (1 — 2/A;)|tfcU^*|. 
We claim that the sequence 

S = tiUi^t2U2^t3U^^ . . . 

fulfills the requirements of the lemma as soon as the sequence {nk)k>i is chosen so that for 
all k, we have 
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1. Uk > n'^] 



2. \tk+iul\\'\ > 

Indeed, condition 1 ensures that no automaton with < k states and of stack alphabet of size k 
can compress tf^u^'' with a ratio better than 1 — 2/k; then condition 2 shows that asymptoti- 
cally the compression ratio of the whole prefix tiu^^ . . . t^u^^ tends to 1 as A; tends to infinity. 
Finally, the third condition ensures that for all the compression ratio of tfc'"fc'°^fe+i'^fc+i 
also tends to 1 as A: tends to infinity. As a whole, for every automaton, the liminf of the 
compression ratio on S is at least 1. 

□ 



E Proof of Theorem 15.1 

In this proof we work with the binary alphabet, the general case can be proven similarly. 
Proof. Let m G N, and let k = k{m),v = v{m),v' = v'{m) be integers to be determined 
later. For any integer n, let denote the set of strings x of size n such that P does not 
appear in x, for every j > k. Since r„ contains {0, l}*"'^^ x {0} x {0, l}'^^^ x {0} . . . (i.e. the 
set of strings whose every /cth bit is zero), it follows that |T„| > 2^*", where a = 1 — 1/k. 

Remark E.l For every string x G Tn there is a string y £ T„_i and a bit b such that yb = x. 

Let An = {ai, . . . Qu} be the set of palindromes in r„. Since fixing the n/2 first bits 
of a palindrome (wlog n is even) completely determines it, it follows that |j4„| < 2^ . Let 
us separate the remaining strings in T„ — An into v pairs of sets Xn^i = {xi^i, . . . Xi^t} and 
Yn,i = {yiA, ■ ■ ■yi,t} with t = '^"2"/"' , {xijy^ = yij for every 1 < j < t and 1 < i < v, 
Xi,i,yi,t start with a zero. For convenience we write Xi for X„^j. 

We construct 5 in stages. Let f{k) = 2k and f{n + 1) = f{n) + v + 1. Clearly 

> f{n) > n. 

For n < A — 1, S'„ is an enumeration of all strings of size n in lexicographical order. For n > k, 
Sn = ai...au . . . xi,t yi,t . . . . . . . . . x,,a^(")+"y.,t • • • 

i.e. a concatenation of all strings in An (the A zone of Sn) followed by a flag of /(n) ones, 
followed by the concatenations of all strings in the Xi zones and Yi zones, separated by flags 
of increasing length. Note that the Yi zone is exactly the Xi zone written in reverse order. 
Let 

c 00 c 1^1 k-\-\ 1 1k — 1 o o 

— blb2 ■ ■ ■ Ok-l I i ...i DfcDfc+i . . . 

i.e. the concatenation of the S'j's with some extra flags between S^-i and 5',^. We claim that 
the parsing of Sn [n > k) by LZ, is as follows: 

ai,...,an, . . . yi^t, . . . . . . . . . l■^^")+'",y^,,t, . . . 
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Indeed after Si, . . . Sk-i l'^ I'^+i . . . l^'^^^, LZ has parsed every string of size < k — 1 and the 
flags l'^ I'^'+i . . . l^fc-i. Together with Remark lE.H this guarantees that LZ parses Sn into 
phrases that are exactly all the strings in T„ and the v + 1 flags . . . , 

Let us compute the compression ratio plz{S). Let n, i be integers. By construction of 
5, LZ encodes every phrase in Si (except flags), by a phrase in Si-i (plus a bit). Indexing 
a phrase in Si-\ requires a codeword of length at least logarithmic in the number of phrase 
parsed before, i.e. \og{P{SiS2 ■ ■ ■ Si-2))- Since P{Si) > \Ti\ > 2"*, it follows 



* 2 na(j — 1) na 

P{Si . . . 5,_2) > ^ 2^^ = > b2<^-^'> 



where h = h{a) is arbitrarily close to 1. Letting ti = \Ti\, the number of bits output by LZ on 
Si is at least 



P{Si) log P{Si . . . Si^2) > k log 62"(*-i) 

> cti{i - 1) 



where c = c(6) is arbitrarily close to 1. Therefore 



\LZ{Si...Sn)\>^Ctj{j-l) 



Since 



\Si... Sn\ = \Si... Sk-il ...l\ + \Sk...Sn\<2^' + ^{jtj + iv + l){f{j) + v)) 

j=k 



and \LZ{Si . . . 5„)| > + YTj=k '^^jU ~ l)i compression ratio is given by 



2^ + z2j=kUtj + {v + l)(/(j) + v)) 



23'= + E-=fc(ji. + (^ + l)(/(j)+^)) 
+ (f -F 

^^"' + E]=k(3'^^+V+mu)+v)) 



_ 2^>' + E]=kih + {v + l)ifij)+v)) 
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The second term in Equation [3] can be made arbitrarily small for n large enough: Let k < 
M < n, we have 



M 

> J^jt, + (M + 1) ^ t, 

j=k j=k j=M+l 

M n n 

j=k j=M+l j=M+l 

M n n 

j=k j=M+l j=M+l 

M n 

>Y^jt,+M Y h + 'i"'' 

j=k j=M+l 



We have 

M n 

j=k j=k 

for n large enough, because f{j) < . Hence 

2'' + E •=fc(ii + + + V)) + E •=fc(i. + + + ^) 

^ — 7- ; T^nrr-. tt > C- 



'2=*^= + E •=fc(ji. + + i)(/(j) + ^)) - ^[2^'= + E "=.(i. + + 1)(/0-) + M 



I.e. 



PLzl-Si . . . 5„) > c - — 
which by definition of c, M can be made arbitrarily close to 1 by choosing k accordingly, i.e 

PLziSx . . . 5„) > 1 - — . 

m 

Let us show that Rp£){S) < ^. Consider the following ILPD compressor C. On any of 
the zones A, Xi and the flags, C outputs them bit by bit; on Yi zones, C outputs 1 bit for 
every v' bits of input. For the stack: C on Sn cruises through the A zone until the first flag, 
then starts pushing the whole Xi zone onto its stack until it hits the second flag. On Yi, C 
outputs a for every v' bits of input, pops on symbol from the stack for every bit of input, 
and cruises through v' counting states, until the stack is empty (i.e. X2 starts). C keeps 
doing the same for each pair Xi, Yi for every 2 < i < v. Therefore at any time, the number 
of bits of Yi read so far is equal to v' times the number of bits output on the Yi zone plus the 
index of the current counting state. On the Yi zones, C checks that every bit of Yi is equal to 
the bit it pops from the stack; if the test fails, C enters an error state and outputs every bit 
it reads from then on (this guarantees IL on sequences different from S). This together with 
the fact that the Yi zone is exactly the Xi zone written in reverse order, guarantees that C is 
IL. Before giving a detailed construction of C, let us compute the upper bound it yields on 
Rpd{S). 



9 



Remark E.2 For any j G N, let pj = ^^(^[l . . . j]) be the output of C after reading j bits of 
S. Is it easy to see that the ratio ^gt^i is maximal at the end of a flag following an Xi zone, 
(since the flag is followed by aYi zone, on which C outputs a bit for every v' input bits). 

\p\ 

Let 1 < t < V. We compute the ratio inside zone Sn on the last bit of the flag 

following Xt+i. At this location (denoted jo), C has output 

n— 1 . ^ 

\pj < 2"^ + J2[j\A,\ + {v + l)(/(i) +v) + ||r, - ^,1(1 + -)] + n\A^\ + iv + l)(/(n) + v) 

j=k 

+ —\Tn - Anlit + 1 + -) 

2v v' 

n-l . 

<2P" + ^[||r,|(i + -)] + |-|r„|(t + i + -) 

j=k 

where p > \ can be made arbitrarily close to ^. 
The number of bits of S at this point is 

n-l 



\s[i . . . jo]| > 5] + n|A„| + -|r„ - An\{t + -; 

j=k 



j=k 

Hence by Remark IE.2I 

, ■ , \Pn\ ^ . , 2^" + EUmii^ + V)] + fJTnlit + 1 + ^) 

lim mf -— 77 < hm mf ; 

™ \S[l...n]\ - n^oo + + 



Since YrjZlj\Tj\ > (n - l)|r„_i| > (n - 1)^, we have 

+ ^|T„|(t + \)> ^|r„| + ^|r„|(t + \) 



j=k 

= ^{v-'- + 2t + h 
2v ^ n 2' 

Therefore 

lim inf — < lim inf ■ 



< lim inf = 

n— »oo 2""" 
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and 

1 ^Um ^ 1 

which is arbitrarily small by choosing v' accordingly, and 



2v ^''-i 



< 



which is arbitrarily small by choosing v accordingly. Thus 

Rpd{S) = liminf j^"^ < J. 

Let us give a detailed description of C. Let Q be the following set of states: 

• The start state gO) and qi,. . .Qw the "early" states that will count up to 

w=\SiS2...Sk-i 1*^ 1^^+^ ...l^'^-^l. 

• Qq,. . . the A zone states that cruise through the A zone until the first fiag. 

• the jth flag state, {j = 1, . . . ,v + 1) 

• q^^ . . . ,q^' the Xj zone states that cruise through the Xj zone, pushing every bit 
on the stack, until the (j + l)-th flag is met, (j = 1, . . . , f). 

Y Y 

• qi q J the Yj zone states that cruise through the Yj zone, poping 1 bit from 
the stack (per input bit) and comparing it to the input bit, until the stack is empty, 
(i = l,...,^). 

• <?o ' • • • ' ^fc which after the jth flag is detected, pop k symbols from the stack that 
were erroneously pushed while reading the jth flag, (j = 2, . . . , u + 1). 

• qe the error state, if one bit of Yi is not equal to the content of the stack. 

Let us describe the transition function 6 : Q x {^,1} x {0, 1} — > Q x {0, 1}. First 5 counts 
until w i.e. for ? = 0, ... u; — 1 

K^lh X, y) = (gj+i, y) for any x, y 
and after reading w bits, it enters in the flrst A zone state, i.e. for any x, y 

S{qw,x,y) = {qo,y)- 

Then 6 skips through A until the string 1*^ is met, i.e. for i = 0, ... /c — 1 and any x, y 



S{Qt,x,y) 

and 



iQi+i,y) ifx = l 
{q^,y) iix = 



S{qk,x,y) = {q{,y)- 
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Once l'' has been seen, 6 knows the first flag has started, so it skips through the flag until a 
zero is met, i.e. for every x,y 



{qi,y) ifa; = l 



where state q^'^ means that the first bit of the Xi zone (a zero bit) has been read, therefore 
S pushes a zero. In the Xi zone, deha pushes every bit it sees until it reads a sequence of k 
ones, i.e until the start of the second flag, i.e for z = 0, . . . A; — 1 and any x, y 

'^^ \iq^\xy) ifx = 

and 

6{qf',x,y) = (go'^y)• 

At this point, S has pushed all the Xi zone on the stack, followed by k ones. The next step 
is to pop k ones, i.e for i = 0, ... A; — 1 and any x, y 

S{<l?^,x,y) = (gS,A) 

and 

S{qk'^,x,y) = {ql,y). 

At this stage, 5 is still in the second flag (the second flag is always bigger than 2k) therefore 
it keeps on reading ones until a zero (the first bit of the Y zone) is met. For any x, y 

On the last step, 5 has read the first bit of the Yi zone, therefore it pops it. At this stage, 
the stack exactly contains the Xi zone written in reverse order (except the first bit), 5 thus 
uses its stack to check that what follows is really the Yi zone. If it is not the case, it enters 
Qe- While cruising through Yi, 5 counts with period v' . Thus for z = 1, ... — 1 and any x, y 

I (ge, A) otherwise 



{q^\\) iix = y 
(ge,A) otherwise 



and 

S{q^>,x,y) 

Once the stack is empty, the X2 zone begins. Thus, for any x,y, 1 < i < v' 

%/,x,zo) = <^ 

[(^0 'O^o) ifa; = 0. 

Then for 2 < j < t> and < i < k — 1, the behaviour is the same, i.e. 



^{(li \x,y) 



{%' ,xy) if a; = 
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and 

At this stage we reached the end of the {j + l)th flag , therefore we quit k bits from the stack, 
and 

At this stage 5 is in the (j + 1) th flag, thus: 

y{(ii A) ifx = o. 

Next the Yj zone has been reached, so for i = 1, . . . — 1 and any re, y 

S{ql',x,y) 

and 



{qe, A) otherwise 



l(ge,A) otherwise 
and for j < u — 1, (5 goes from the end of Yj to Xj+i i.e. for any 1 < i < v' 

[(^0 jO^o) if X = 0. 
and at the end of y^, a new A zone starts, thus for any 1 <i<v' 

iq1,zo) ifx = l 
{q^,zo) ifx = 0. 



n ^ J (^I'^o) if X = 1 
6{q-\x,ZQ) - 



Once in the q^ state, S never leaves it, i.e. 

S{qe,x,y) = {qe,y) 

Y Y 

The output function outputs the input on every states, except on states qi\ ■ ■ ■ , qj 
{j = 1, . . . ,v) where for 1 < i < -y' 

I'iQl' ,b,y) = X 

and 

^{(lv',b,y) = 0. 



□ 
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